<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <title>The Twisted Plugin System</title>
  </head>

  <body>
    <h1>The Twisted Plugin System</h1>

    <p>The purpose of this guide is to describe the preferred way to
    write extensible Twisted applications (and consequently, also to
    describe how to extend applications written in such a way).  This
    extensibility is achieved through the definition of one or more
    APIs and a mechanism for collecting code plugins which
    implement this API to provide some additional functionality.
    At the base of this system is the <code
    class="API">twisted.plugin</code> module.</p>

    <p>Making an application extensible using the plugin system has
    several strong advantages over other techniques:</p>

    <ul>
      <li>It allows third-party developers to easily enhance your
      software in a way that is loosely coupled: only the plugin API
      is required to remain stable.</li>

      <li>It allows new plugins to be discovered flexibly.  For
      example, plugins can be loaded and saved when a program is first
      run, or re-discovered each time the program starts up, or they
      can be polled for repeatedly at runtime (allowing the discovery
      of new plugins installed after the program has started).</li>
    </ul>

    <h2>Writing Extensible Programs</h2>

    <p>Taking advantage of <code class="API">twisted.plugin</code> is
    a two step process:</p>

    <ol>
      <li>
        <p>
        Define an interface which plugins will be required to implement.
        This is done using the <code class="API">zope.interface</code>
        package in the same way one would define an interface for any other
        purpose.
        </p>

        <p>
        A convention for defining interfaces is do so in a file named like
        <em>ProjectName/projectname/iprojectname.py</em>.  The rest of this
        document will follow that convention: consider the following
        interface definition be in <code>Matsim/matsim/imatsim.py</code>, an
        interface definition module for a hypothetical material simulation
        package.
        </p>
      </li>

      <li>
      At one or more places in your program, invoke <code
      class="API">twisted.plugin.getPlugins</code> and iterate over its
      result.
      </li>
    </ol>

    <p>
    As an example of the first step, consider the following interface
    definition for a physical modelling system.
    </p>

    <pre class="python">
from zope.interface import Interface, Attribute

class IMaterial(Interface):
    """
    An object with specific physical properties
    """
    def yieldStress(temperature):
        """
        Returns the pressure this material can support without
        fracturing at the given temperature.

	@type temperature: C{float}
	@param temperature: Kelvins

	@rtype: C{float}
	@return: Pascals
        """

    dielectricConstant = Attribute("""
        @type dielectricConstant: C{complex}
        @ivar dielectricConstant: The relative permittivity, with the
        real part giving reflective surface properties and the
        imaginary part giving the radio absorption coefficient.
        """)

    </pre>

    <p>In another module, we might have a function that operates on
    objects providing the <code>IMaterial</code> interface:</p>

    <pre class="python">
def displayMaterial(m):
    print 'A material with yield stress %s at 500 K' % (m.yieldStress(500),)
    print 'Also a dielectric constant of %s.' % (m.dielectricConstant,)
    </pre>

    <p>The last piece of required code is that which collects
    <code>IMaterial</code> providers and passes them to the
    <code>displayMaterial</code> function.</p>

    <pre class="python">
from twisted.plugin import getPlugins
from matsim import imatsim

def displayAllKnownMaterials():
    for material in getPlugins(imatsim.IMaterial):
        displayMaterial(material)
    </pre>

    <p>Third party developers may now contribute different materials
    to be used by this modelling system by implementing one or more
    plugins for the <code>IMaterial</code> interface.</p>

    <h2>Extending an Existing Program</h2>

    <p>The above code demonstrates how an extensible program might be
    written using Twisted's plugin system.  How do we write plugins
    for it, though?  Essentially, we create objects which provide the
    required interface and then make them available at a particular
    location.  Consider the following example.</p>

    <pre class="python">
from twisted.plugin import IPlugin
from matsim import imatsim

class SimpleMaterial(object):
    implements(IPlugin, imatsim.IMaterial)

    def __init__(self, yieldStressFactor, dielectricConstant):
        self._yieldStressFactor = yieldStressFactor
        self.dielectricConstant = dielectricConstant

    def yieldStress(self, temperature):
        return self._yieldStressFactor * temperature

steelPlate = SimpleMaterial(2.06842719e11, 2.7 + 0.2j)
brassPlate = SimpleMaterial(1.03421359e11, 1.4 + 0.5j)
    </pre>

    <p><code>steelPlate</code> and <code>brassPlate</code> now provide both
    <code class="API">IPlugin</code> and <code>IMaterial</code>.  All that
    remains is to make this module available at an appropriate location.
    For this, there are two options.  The first of these is primarily useful
    during development: if a directory which has been added to <code>
    sys.path</code> (typically by adding it to the <code class="shell">
    PYTHONPATH</code> environment variable) contains a <em>directory</em>
    named <code class="shell">twisted/plugins/</code>, each <code
    class="shell">.py</code> file in that directory will be loaded as a
    source of plugins.  This directory <em>must not</em> be a Python
    package: including <code class="shell">__init__.py</code> will cause the
    directory to be skipped and no plugins loaded from it.  Second, each
    module in the installed version of Twisted's <code class="shell">
    twisted.plugins</code> package will also be loaded as a source of
    plugins.</p>

    <p>Once this plugin is installed in one of these two ways,
    <code>displayAllKnownMaterials</code> can be run and we will see
    two pairs of output: one for a steel plate and one for a brass
    plate.</p>

    <h2>Alternate Plugin Packages</h2>

    <p><code class="API">getPlugins</code> takes one additional
    argument not mentioned above.  If passed in, the 2nd argument
    should be a module or package to be used instead of
    <code>twisted.plugins</code> as the plugin meta-package.  If you
    are writing a plugin for a Twisted interface, you should never
    need to pass this argument.  However, if you have developed an
    interface of your own, you may want to mandate that plugins for it
    are installed in your own plugins package, rather than in
    Twisted's.  In this case, you probably also want to support <code
    class="shell">yourproject/plugins/</code> directories for ease of
    development.  To do so, you should make the <code
    class="shell">__init__.py</code> for that package contain at least
    the following lines.</p>

    <pre class="python">
from twisted.plugin import pluginPackagePaths
__path__.extend(pluginPackagePaths(__name__))
__all__ = []
    </pre>

    <p>The key behavior here is that interfaces are essentially paired
    with a particular plugin package.  If plugins are installed in a
    different package than the one the code which relies on the
    interface they provide, they will not be found when the
    application goes to load them.</p>

    <h2>Plugin Caching</h2>

    <p>In the course of using the Twisted plugin system, you may
    notice <code class="shell">dropin.cache</code> files appearing at
    various locations.  These files are used to cache information
    about what plugins are present in the directory which contains
    them.  At times, this cached information may become out of date.
    Twisted uses the mtimes of various files involved in the plugin
    system to determine when this cache may have become invalid.
    Twisted will try to re-write the cache each time it tries to use
    it but finds it out of date.</p>

    <p>For a site-wide install, it may not (indeed, should not) be
    possible for applications running as normal users to rewrite the
    cache file.  While these applications will still run and find
    correct plugin information, they may run more slowly than they
    would if the cache was up to date, and they may also report
    exceptions if certain plugins have been removed but which the
    cache still references.  For these reasons, when installing or
    removing software which provides Twisted plugins, the site
    administrator should be sure the cache is regenerated.
    Well-behaved package managers for such software should take this
    task upon themselves, since it is trivially automatable.  The
    canonical way to regenerate the cache is to run the following
    Python code:</p>

    <pre class="python">
from twisted.plugin import IPlugin, getPlugins
list(getPlugins(IPlugin))
    </pre>

    <p>As mentioned, it is normal for exceptions to be raised
    <strong>once</strong> here if plugins have been removed.</p>

    <h2>Further Reading</h2>

    <ul>

      <li><a href="components.xhtml">Components: Interfaces and Adapters</a></li>

    </ul>

  </body>
</html>
